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ABSTRACT " * . v . 

^ Although there is high intetest in ^determining 
whether or not an educaticmal program provides good value for its 
cost, it is difficult tq make this determination, since people ar^ • 
not generally conceptualized as products and since educational 
benefits are not easily translated, into financial terms. Economic 
pr.inGiples suggest that the co3t of obtaining soaething is the value 
placed on. whatever must be sacrificed to 'obtain it. This opportunity 
cost varies among individuals, changes with time, atid depends upon 
the 'in dividual 'Is prior learning. The value of learning can be ' . 

computed by approximating the banker* s compound intterest form^ula; the 
opportunity cost and the number of times the liearning is used are 
considered in this formula. Once the costs are computed, the . 
cost-benefit* model shcJkld consider the service provided, as well as 
all those who benefit from it and those who pay for it, A Ithough • this 
procedure has not b^en* empiri'call y validated, it provides a simple v 
and objective method of examining multiple outcomes over time.. (Ten 
steps for carrying out a cost-benefit, evaluatitjn are described). 
(GDC) . 
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A Cos|-Benef lt Methodology for Summatlve Evjaluatlon 



Oa^Id Ciiurchman 



Orta of tl^e mcrst fr^i^r^--distlnctT3l^ by evaluators Is that between 

foriT^tf^ve aft<^ s ununa^fve" eval uat ion {Scr^veit The fonnef^ usual l%- 

thougtit of In Velationship to , ma kltig . Improvements in an ^ducattonal program;- 
the latter with determining Whether or not to adopt a particular program 
)Fon some purpose. St^ke (I967) points out that formative evaluation is of ■ 
Interest to authors, develapers and publ fshers, whi le summatWe evaluation 
Is oriented to teacliiers, administrators and" consumers. , ^ 

One. of the major factor^' that teafhers, administrators or consumers 
are interested in Js whether or not an educational program provides good 
va 1*00 for the^ money spenf on it. Thjs Js difficult enough in any clrcum-* 
stance— for example, selecting thfe^best tire fof^ the type of car you hav« 

* *■ . * ■ 

and' the type of dl-lvlng-ypu do, where thieWiteria are re)atively few, 

■_ • V* . ( , ^ ■ . • .■ , ■' ^ 

relatively clear and performance of the product', <;an be predicted with reas- 

• ♦ I ^ ■ 

on^ble accuracy. But, It is exceedingly difficult In the Cjrfse of an educa- 
tlonal program. There us^al 1y is(1lttle difficulty In deteftnlnlng the costs 
of providing the program with reasonable accuracy, but ft often is Cinclear 
Just what Jtlte benefits dre and even l^ss clear how to value them. 

there are two important- reasons for this difficulty. First, educators 
often object tc^thipking of people a$ "products,*' preferring instead to 
speak In terms such as "the full potential of tlie Individual." Despite a' 
difference in language, neither educators nor economists want to see 



educational ^resources wasted. , Both tend to, a§ree th^t a mofe efficient 
way of educating Is preferred to a less eff Tcient one, providing that 
cultural and social norms are ^aken iriito account In determining efficl6ncy» 
The second difficulty Is that of translating educational benefits IntoX 

• ... - . ■ ^ 

financial terits in some objective fashion., Schultz, pne of the pioneers 
in quantitatjvle measurement of returijs to' investment in education, demon- « 
strated that it^ is necessary to take education into account In explaining 7 
economic growt^ OS(>dT^. But, Schultz, studied education only as it corr~ - 
tVibuted to the production of goods and. services. Educators would^gree ; 
that this' is one purpose of education, but not that it is the only purpose. 

^^Jhe real difficulty lies In determining a cash value for edqcation that 
contributes to quality of life for £fie individual. Just*«that Is It worth 
in dol lars ^to "learn to enjoy a Rembrandt? Or to be able to play soccer? Or 
to know the language of a country when travel 1 Ing abroiad? Each of these 
wil^ enable some very small numberxof peopre to earn their living. Each 
will enlarge GNP 'in sojb* small degree by creating a market for/certain 
typks of products, and services. But, for most people the %hrte types of ^ 
learning are non-*economi<. ' How can a money value 'be detemf Red for each, 
and for similar type^ of , learning, in some replicat>fe and objective fashion? 
The Cash Value of Educational Benefits ^ 

The solution ties ih the economists' l^ea >6f opportunity costs. The cost 

\ '■ - * 

/ * * ■ 

of obtaining anything is the value placed on whatever must be sacrificed to 

obtain it (Heyns, 1973). T*»e value of havi.ng learned' to *njoy a. Rembrandt 

"> ■ ■ " 

is what it woul(jl cost to induce* the individual to forego the opportunity to 

/V * . ■ • ' O' 

see^ a- Rembrandt. This cost w|J I vary in three important ways. 



'First, opportunity cOst for any product or service wl 1 1 vary among Indl 
viduals. Therefore, It Is necessary to determine an average opportunity 
co^t among all Individuals who have completed whatever program Is being . 
evaluated. This can be done by sampling students at the end of the course 

and determining the opportunity costs by presenting them with various 

*" ■ Si. * 

hypothetical situations. The assumption fs*that the more successful the 
program has been the higher the opportunity costs associated with Its goals 



wl.n be. 

/ * ■. ■ ■ • 

Second, opportunity costs will vary with tline. Some learning becomes 

obsolete Very quickly. This Is true In rapidly developing fields such as 

computer technology, -and it Is true of llesure time, fads, such as roller 

skating, disco dancing and* roller disco, very, much In fashion as this Is 

written, but hopeful^ly sufficiently outmoded by the time this Is read to ^ 

prove the point. Other ll&sure act'lvltles have more lasting appeal but 

require physical stamina, so that there ^Is a tendency to switch from 

active participant to Spectator then to lose Interest entirely. Soccer 

might be ^n example'. Other llesure activities', particularly cultural 

ones such as appreciation of art, tend to fncrease rather thaft diminish 

with time. It Is necessary to estimate, for* each educational outcome, 

t ^^^^^ 

whether It will Increase oV decrease over time In Its effect on the 
Individual^ and It; Is necessary to^^stlmate the "service life" of the 
learning. ' , ^ , 
Third, the opportunity for an Individual to make use of different 

» 

types learning wl II vary considerably. The IndlvMual who has learned 
to erf joy Rembrandt, enjoy soccer and speak a foreign language wlM not 



do|l11 three every day. Some esttjnate of the fre(^ncy that each educational 
outcome will prove useful to an individual Is equally Important in determinfny 
its valuer A reasonable way of making dn estimate for each type of learning 

" ■ V r 

«iu$t be made, there being no simple way that will provide a suitable average 
for every 'conceivable educational outcome* ^ 



ICnowing the cu;*rent value of an educational benefit in terms of Its 
opportunity co$|ts/thc frequency that It |s. useful to prog i;am graduates » 
Its service life and whether It increases, or decreases in value With time, 
It is possible to-compute the vaVue of the learning by approprfatlng the 
compound interest formula of the banker: 



^ ■ ■ .±n 



V « 0(1 4- .f/3^5) _ ^ 

that Is, the th^ value of learning equals pppbrtunity cost times the quantity 

o^e pl.us\the nujiiber of times it is used in a year divided by the number of 

fdays In a year jpfalsed to the power of the service life of the learning. Pos- 

/ itlVe values of' n are used if the benefits increase with tlrne. Just as the 

banker uses a pjjsltlve value of n (we hope) In calculating our bank balances. 

Negative values; of n are used If the value of the learning declines with 

time, just as ^he banker uses a negative value of n la calculating your loan 

balance. V 

Nt is unlikely that opportunity tosts and compound interest formulas ^ 

'■>■■.. ■ ' ' ' • 

wMl provide^ definitive method for calculating the long-term value of 

educational benefits. But» it does provide a starting point for finding 

better ones, iind until they are found, it does provide a repllcable and 

objective m«|liod that will, permit comparisons among vastly different types 

of leamingjil Presumably suggestions for Improving the formulas that 

* • • ,1" i 



have an empirical Vather than a rhetocical base, wj 11 prove the more valOabI 
approach In the long run. ^ 
A Cost-Benefit Model 



The preceding tethnique makes it possible to assign a cash value to any 

type of education^ henefft, the. problem that has plagued those interested 

- ■ - * '' ' ' '■ 

In cost-benefit models, for education. But, It* does not provide a complete 

solution to the question that concerns Stake's jteachers, ^administrators and 

consumers. Th<i real question is whether the. benefits warrant the costs 

required to achieve them. 

The key to solving this problenf Is-recognltlon th^t every benefit to 

some group or grpups Is a cost to some grqup or groups. Students may take 

a course for which they pay tuition, but the tultioji may cover only 2/3 of 

the costs, the difference being made up by some other group, such as tax*^ 

• * 
payers or alurnnl* Benefits accrue to one group, ^students; costs are met 



by two groupSf students and taxpayers* If r^^cessary, the groups c^uld 
hd\fe Jieen distinguished more precisely* For example j students could- have 
been subdivided into art majors and other majors, full-time students and 
extension students, or whatevdfr divisions ar^urequired to answe^ the 
questions b^ing asked about the program* 

This approach makes it possible to develop a complete cost-benefit 
model for any educational program of any size/ from a single, class to ' 
local^ state or even national school system (the resources required to 
complete each varying considerably of course). Tt^ree types of information 
are heeded: benf icij^ries, payers, and services provided. A very convenient 
way both to develop and to display such a model Is a chart sueh as that in 
Figure 1. Columns are headed with beneficiaries, in whiitever detail Is • 



required* Rows are headed* with payers. To insure completeness^ column 
and row ^ head ings%ust be iclenticaK. . > • ^ , - 

m 

Figure 1 

Sample Program Cost-Benefit Model* 
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Cells are comp1et;ed by. entering educatlonaJ services. A general evaluation 
of an eyritlre school would include a vas*t and diverse ninpber of services, only: 
some educational/ In addition, athletic, cultural and social events would be 
listed, as well as commnnity sefvices such <eis use of .recreational' and/library 
faciiities, provision of facilities for public meetings, land at the university 
level ,• research, consulting ancf many other services. VCells may* have any number 
of entries (indicating several services paid for by the same group and receive*^ 
by the 'same 'beneficiarl^es) . The same servi^ce may be entered In several cells 

(indicatin^g that more than one payer met the cost, or" that more than one group 

J* ■ ' ' ' • . ■ 

benefitted, depending on whether the multiple entry is in a column, or i^row).- 

Some bel Is may remafn empty indicating no service paid for or received py that 

• • • *- * 

\ - ■ ' . ■ ■ ' V-^v 



^Adapted from Caffrey and' Isaacs (1970 
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combination 6fj payer and beneffqlary. -Much of the success of an evaluation 

will depend oijt the thoroughness with which ail services provided by tlie ' 

program .being evaluated are identified. 

Once the cost-benefit model is established. It is necessary to determine 

values for each entry* Program costs can bie determined directly from 
»■ ^ ' ■ • 

organizational records following tj^t organization's standard accounting 

procedures. Program benefits can be determined on the basis of opportunity 

costs and the formula suggested above. Costs will equal benefits only In 

the most unusual circumstance. This Implies an unequal exchange has taken 

plac*, wKlch Is the only circumstance In which an economic exchange ei^«2» 

takes place. (Heyns, ' 1973) . If two Individuals or groups excl^nge gqodi or 

services, they <^o so only because each values 'what Is being surrendered 

Uss than what Is being obtained. Viewed from either side, the exchange Is 

unequal, bu^ both' sides have more than theV did before In terms of their 

• r 
own needs, so are better off . than they were before the (0x<^hange took place.' 

Once values are determined. It Is possible to compare any combination of 

/ \ 

benefits with any combination of costs*of Interest to the audience for the 
evaluation. Efforts to reduce these figures to a single Index, as statisticians 
using research designs often do, are Inappropriate, because no single one can 
tell the full story or answer every question thdt may be raised. Ratios of 

benefits to costs may be calculated for purposes of comparison If desired. 

*• ' ' 

'■ t- ' 

In special circumstances, when It Isdeslred to maximize the combined ^ 
total learning, and the costs ijust be kept within specified constraints^ It 
Is possible to construct equation? from the cost-benefit model of Fl*gure I , and 
through linear programming to determine the maximum potential of the program 
* being evaluated. Just as no engine ever achieves It? maximum because of 



friction, so^no educational systenupver wil 1 achieve Its theoretical potentlalT 
But-, just as an Imaginary frictlonless engine provides a standard of comparison 
for real engines, so -the extent to which the theoretical maximum Is achieved by 
a particular educational, program provides a means for Judglne real educational 
-programs. Eventually this may permit develo'pment of standards such as those 
iloj^provided for Interpreting statistical tests. 

A step-by-step methodology for carrying out an evaluation based on; these 
principles Is provided below, followed by* a llstl^'of the* advantages and 

dlsdavantages of the approach. • ^ *^ 

l ' 

Steps for Carrying Out a Cost-Benefit Evaluation ^ 

2. Isolate the geogrqphio vegion to be studied, Th6 evaluation must be ^ 
IJmlted^ to a reasonable area In terrtls of th*^ resources available to conduct 
the evaluation and, more Important, one credible to those whose Interests will 
be affected. Natural geog raph I cc boundaries, school or tax district boundaries 
and school .service areas based on residence of students and staff can be 
considered In Identifying the geographic reglop to which the evaluation will * 
be limited. The region selected should be outlined on a map. If the boundaries 
are likely to be disputed when results are published, then public comment and 
adjustment should take place before the study begins. 

2. Identify payers and henefioiaries^ ^ chart similar to that In Figure 1 

I 

must be constructed with Identical Column and row headings consisting of the 

groups tnthe geog raph I C| reg I on that Incur costs or receive Refits from the 

program being evaluated. It l,s vital to anticipate at this po-lnt the level 

. ! ■ .. / • " * 

of dc|tall required during analysis of the data, so that al Ir groups -of Interest 

! . - • « - 

will be' named on the chart. * ' ' 



^ • .•: . ■ , 

3. Identify aerviGea provided hy the prograhi being -evaluated, tell, 
entries Indicate who pays (by reading the row heading) for and who benefits 
♦4by reading the column heading) from the seVvlce named. There Is no limit 

to the number of entries that can be made in any one cell and no requirement 

' - . *' • ■ 
that there be an entry in any particular cell. It may be nfecessary to make 

the same entry In several cells to Indicate multi pie beneficiaries, multiple 

payers or both. It Is essential to the success of the evaluation that all 

services of the program be Identified.' Usually the most appropriate method. 

Is a brainstorming session In which key staff having even remote connections 

with the program participate. ' * ' 

'I 

4. ^Construct benefit and cost equations. Benefits to any group consist 
of the sum of benefits In any giVen column. Costs to any group consist of 
the sum of costs In any given row. El thereto reduce over- long equations or 

» 

to develop logical ones with respect to questions being asked about the pro- 
gram, ttrese may be separated into as° many equations as appropriate. 

5. Eatimate frequency that aervice ia used by graduatea .^^^me appropriate 
means of estimating how often the service Is, useful each year on average to 
program graduates must be found. In many cases, local or national demographic 
Information will provide the answer. The reader not famMlar with the wealth 
of Information exlstiq^ without tht Deed for conducting another survey should 
contact a regional office of the U.S. Census Bureau, a major unlvferKty library 
where this Ihformatlon Is available^ or consult the very abbreviated summaries 
in the annual Statiatical Abstract of the United States or The Statiatical 
Hiatory of the United' Statea. (yS7f>) . 

6. Eatimate aervice life of benefita and amortization period of coats. 

<. ■ . ' ■ 

Each benefit will be valuabl^e for a varying amount of time, and this period 



must be estimated for each bWefit. Some benefits have U fe'-tiin^\vljlue. 



which dan be estimated from Hfe expectancy tables. Ma^l^ediicatlonal c^sts 
will be ahnualt but some capital costs may be anx)rtized over a longer • . 
period, usually by paying off bohds that simplify the task of estimating the 
period Involved^ . : ' ' - \. 

?\ Determine whether' bene fits increase or decrease with time. Skills * 
or kpowledge that becomes' obsolete or Is forgotten are defined as decreasing 



ir^ater 



0 



with time; those that do no^ become obsolete or become greater through use 
are defined as" Increasing wl th time. In making use of the -formula (Step 9). 
the exponent is given a 'negati ve .s Ign for dec red's I ng skills, and a positive 
.sign for increasing ones. ' ^ 

"8. Determine (ciarrent values hosta^anS, benefits^ Tb^ current "value 
for e4ch cell entryrmust be determined both jb$ '^a cost ?nd as a benefit. 
Institutional records will provide thfe required cost estimates. The concept 
of opportuni ty cost discuss^ed above" (page 2-3^ can be used to determine 

. ■ , . t 

values for entries for which di rect'^lnfom^Hpon on value is^not available. 

A representative samplc^^^ program students must be obtained, and they must 

be given a questionnaire asking them to indicate how much they would have 

to be paid to forego program-rielated opportunities. ' . ' 

9. Determine long-teym costs and benefits. The formula V « 0 (I f/^SS) 
Is^usjsd as discussed above (page h) . Value for various time periods can be 

calculated/ and ratios of benefits to costs dJIermined^ 

20. Prepare evaluation report. A report of the evaluation should be written 

so that it will* answer a wide range of '.questions as tcJ^whp benefits at What 

cos t» from the program under evaluation. Every ef fort "^should be made^to make 

the report as rea^ab-le as possible and care should be taken to insure that 

- • 

the detail provided is appropriate to the audience that wili receive the 
report. 



±n 



4 . ' • • ■ • 1 



10 

Advantages of the flethodology . ' ^ 

< * ' ■ t 1 • ■ 

* The methodology -descrived above has five significant advahtag^ over other 

: approaches to conductlhg a sunmatlve^valtiation: * ^ ^ ' 

« 1. It provl'des an objective w£^ of estimating ediicat Idh^ benefits in 

' monetary terms," . \ ' * * • ^ * 

2. , it provides an^* approach tb Evaluating the long-term effects of ^ 

_* • ,l?ducat_loh. 



It is^not.Mmlted to behaviorai^objecti ves/ but addresses multiple 
, outcomes of educattpn. ^ * V * 

Results are presented in terms that people can understand without 
special training, rather .than in the esoteric statistical language 
of many evaluation reports. , ' 

5. it provides a starting, point for improved methodologies. 
Disadvantages of the Methodology * \ 

Compensating for these advantages are twp significant ^sadvantages: ' V 

1. There Is no empirical basis for assuming the formula Is appropriate, 
or that educational effects are compouikled annually. ^ 

2. Thete is no equivalent to statistical tables and their underlying 
probability distributions tp fadilitate interpretation of results. 
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